Online and Stochastic Universal Gradient Methods for Minimizing Regularized H'older Continuous Finite Sums

摘要

Online and stochastic gradient methods have emerged as potent tools in largescale optimization with both smooth convex and nonsmooth convex problems fromthe classes $C^{1,1}(\reals^p)$ and $C^{1,0}(\reals^p)$ respectively. Howeverto our best knowledge, there is few paper to use incremental gradient methodsto optimization the intermediate classes of convex problems with H\"oldercontinuous functions $C^{1,v}(\reals^p)$. In order fill the difference and gapbetween methods for smooth and nonsmooth problems, in this work, we propose theseveral online and stochastic universal gradient methods, that we do not needto know the actual degree of smoothness of the objective function in advance.We expanded the scope of the problems involved in machine learning to H\"oldercontinuous functions and to propose a general family of first-order methods.Regret and convergent analysis shows that our methods enjoy strong theoreticalguarantees. For the first time, we establish an algorithms that enjoys a linearconvergence rate for convex functions that have H\"older continuous gradients.